Conversation
Introduce a fast local hook layer that shifts cheap, deterministic checks left without replacing CI: - .pre-commit-config.yaml: hygiene fixers + Ruff lint(--fix)/format at commit; fast mocked unit tests at pre-push; opt-in local mypy hook (off by default). - pyproject.toml: [tool.ruff] (E,W,F,I; E501 owned by the formatter; py38) and a [project.optional-dependencies] dev extra so `pip install -e .[dev]` bootstraps the toolchain. - CI: new `lint` job runs `pre-commit run --all-files` so hooks are enforced even when skipped locally; model-download integration tests stay CI-only. - CONTRIBUTING.md: dev setup + how the commit/pre-push/CI layers relate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One-time mechanical pass to satisfy the newly added pre-commit hooks. No functional changes: - Ruff format + import sorting (isort) across src/, tests/, examples/. - Ruff autofixes: remove unused imports, collapse multi-imports; drop dead local assignments (conftest esm1 `device`, example script vars). - Mark the intentional `pepe.api.embed` re-export in __init__.py with noqa. - Generic fixers: trailing-whitespace + final-newline normalization on markdown/json/txt/yaml/workflow files. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Raise the floor to Python 3.10 so packaging metadata, CI, and the type/lint tooling all agree (mypy and Ruff already targeted 3.10): - pyproject.toml / setup.py: requires-python >=3.10, drop 3.8/3.9 classifiers. - .github/workflows/test.yml: unit matrix now [3.10, 3.11]. - .github/conda/meta.yaml: python >=3.10 (host + run). - pyproject [tool.ruff]: target-version = "py310". - CHANGELOG: note the removal under [Unreleased]. Bumping the Ruff target to py310 lets the formatter use parenthesized `with` statements (3.10+ syntax); reformats four files accordingly. No behavior change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add pre-commit hooks + drop Python 3.8/3.9
Ship a CPU Docker image alongside PyPI/Conda releases so users can run PEPE without a local Python setup. Co-authored-by: Cursor <cursoragent@cursor.com>
… release to after Docker publish
Add GHCR Docker publishing to test and main release workflows
Enables metl-*-1d model dispatch, METLEmbedder/METLDataset, typed errors for missing package and 3D models, CI integration test behind METL_TEST, and docs in CHANGELOG (1D-only; no logits/attention). Co-authored-by: Cursor <cursoragent@cursor.com>
Quick start install from GitHub [metl] extra, CLI example, supported models, and limitations aligned with CHANGELOG. Co-authored-by: Cursor <cursoragent@cursor.com>
Document pepe-cli[metl] and direct metl-pretrained install, plus CLI/library usage for metl-g-20m-1d. Co-authored-by: Cursor <cursoragent@cursor.com>
Address review findings on the METL 1D embedder: - _register_repr_hooks: layer 0 now captures the encoder input embeddings (via a forward_pre_hook) instead of aliasing to layers[-1]; the final-layer norm hook is guarded so norm-less METL models fall back to the last block instead of crashing on None.register_forward_hook. Collapses the duplicated make_hook closure into one definition. - _load_layers: default (layers is None) now returns all transformer layers, matching the HuggingFace embedders, instead of only the final layer. The batch-dimension review item was verified against metl-pretrained source (batch_first=True throughout) and needs no change. Adds test_metl_hooks.py (deterministic, no metl download) covering the hook mapping and layer defaults, and extends the gated integration test with batch-consistency and multi-layer checks.
Add METL 1D embeddings (Option A)
…dency PyPI/TestPyPI reject packages with direct URL dependencies (PEP 440). Remove `metl-pretrained @ git+https://...` from [project.optional-dependencies] in pyproject.toml and from extras_require in setup.py. Users should install metl-pretrained directly from GitHub. Update README, CHANGELOG, and error messages accordingly.
…sh-test Fix TestPyPI publish failure: remove git URL from metl optional dependency
Cut the Unreleased section into [1.5.0], covering METL 1D embeddings and GHCR Docker publishing. Restores the Docker/GHCR changelog entry dropped during an earlier merge into test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR cuts release 1.5.0 and ships a new optional METL 1D embedding backend, alongside updated publishing/release automation (PyPI/Conda/GHCR) and a minimum Python version bump to 3.10.
Changes:
- Add METL 1D model dispatch +
METLEmbedder/METLDataset, plus mocked unit tests and an opt-in integration test. - Add/extend release automation: pre-commit lint CI, Docker image build/publish to GHCR, and updated trusted publishing workflows.
- Drop Python 3.8/3.9 support (packaging + CI + conda) and document the dev toolchain (
.[dev], Ruff, pre-commit).
Reviewed changes
Copilot reviewed 40 out of 50 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| src/tests/test_sync_arguments.py | Minor formatting for API/CLI drift-guard test. |
| src/tests/test_streaming_roundtrip.py | Formatting + clarity tweaks in streaming roundtrip regression tests. |
| src/tests/test_splitting.py | Import cleanup + formatting in splitting/reconstruction integration tests. |
| src/tests/test_reconstruct_mean_pooled.py | Minor formatting/whitespace. |
| src/tests/test_model_selection.py | Test formatting + import ordering updates; keeps model-selection assertions. |
| src/tests/test_metl_integration.py | New opt-in METL integration tests (skipped unless METL_TEST=1). |
| src/tests/test_metl_hooks.py | New unit tests for METL layer-selection and hook-capture semantics. |
| src/tests/test_metl_dispatch.py | New unit tests for METL model-name dispatch + typed errors. |
| src/tests/test_generic_safeguards.py | Refactor patch blocks / formatting; maintains generic safeguards coverage. |
| src/tests/test_generic_hf_integration.py | Minor import ordering tweak for integration test. |
| src/tests/test_esmc_modes.py | Formatting for long literal sequence. |
| src/tests/test_device_logic.py | Formatting + import ordering; retains device-selection assertions. |
| src/tests/test_custom_embedder.py | Minor formatting/whitespace. |
| src/tests/test_api_unittest.py | Formatting + minor cleanup in API tests. |
| src/tests/conftest.py | Minor import ordering cleanup in fixtures. |
| src/pepe/utils.py | Add METLDataset and reformat/clean up existing utilities. |
| src/pepe/model_selecter.py | Add METL model dispatch + validation; keep local-path precedence. |
| src/pepe/model_errors.py | Add typed METL errors (METL3DNotSupportedError, METLPackageRequiredError). |
| src/pepe/embedders/metl_embedder.py | New METL embedder implementation (metl-pretrained backend). |
| src/pepe/embedders/huggingface_embedder.py | Import/format refactors; no intended behavior change. |
| src/pepe/embedders/esm_embedder.py | Import ordering/formatting only. |
| src/pepe/embedders/custom_embedder.py | Import ordering + minor formatting; no intended behavior change. |
| src/pepe/embedders/base_embedder.py | Formatting refactors; no intended behavior change in embedding loop. |
| src/pepe/api.py | Formatting + minor signature punctuation; no intended behavior change. |
| src/pepe/main.py | Minor formatting/whitespace. |
| src/pepe/init.py | Version bump to 1.5.0 and explicit re-export comment. |
| setup.py | Python>=3.10, remove 3.8/3.9 classifiers, add esm extra. |
| requirements.txt | Whitespace normalization. |
| README.md | Document METL backend install/use + update supported model list/docs. |
| pyproject.toml | Python>=3.10, add optional-deps (esm, dev), add Ruff config. |
| notebooks/verify_embedder_consistency.ipynb | Formatting-only edits in notebook cells. |
| examples/model_selection.md | Trailing whitespace fix. |
| examples/embedding_options.md | Whitespace/formatting fixes. |
| examples/custom_model/example_protein_model/vocab.txt | Ensure newline/format consistency. |
| examples/custom_model/example_protein_model/vocab.json | Ensure newline/format consistency. |
| examples/custom_model/example_protein_model/tokenizer_config.json | Ensure newline/format consistency. |
| examples/custom_model/example_protein_model/special_tokens_map.json | Ensure newline/format consistency. |
| examples/custom_model/example_protein_model/config.json | Ensure newline/format consistency. |
| examples/custom_model/create_example_custom_model.py | Import ordering + minor cleanup of unused returned paths + print formatting. |
| Dockerfile | New minimal runtime image build for CLI distribution. |
| CONTRIBUTING.md | Switch setup guidance to pip install -e ".[dev]" + document pre-commit workflow. |
| CHANGELOG.md | Cut 1.5.0 release notes and list major additions/removals. |
| .pre-commit-config.yaml | New pre-commit configuration (Ruff lint/format + staged unit tests). |
| .github/workflows/test.yml | Add lint job (pre-commit), update Python matrix to 3.10/3.11, add METL tests. |
| .github/workflows/publish-test-branch-trusted.yml | Add GHCR publish for test branch + formatting cleanups. |
| .github/workflows/publish-main-branch-trusted.yml | Add GHCR publish + release improvements (artifact upload/download). |
| .github/QUICK_SETUP.md | Whitespace/formatting fix. |
| .github/conda/meta.yaml | Bump conda Python requirement to >=3.10. |
| .github/bioconda_submission_guide.md | Whitespace/formatting fixes. |
| .dockerignore | New dockerignore to reduce build context and exclude large/dev artifacts. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+71
to
+74
| split_long_sequences=self.split_long_sequences, | ||
| ) | ||
| self.special_tokens = torch.tensor([0], device=self.device, dtype=torch.int8) | ||
| self.layers = self._load_layers(self.layers) |
Comment on lines
+219
to
+224
| def _handle_sequence_splitting(self, max_allowed): | ||
| """Split sequences without reserving space for BOS/EOS tokens.""" | ||
| new_sequences = {} | ||
| self.chunks_mapping = {} | ||
| chunk_size = max_allowed | ||
| overlap = self.split_overlap |
Comment on lines
+40
to
+46
| if self.return_logits: | ||
| logger.warning( | ||
| "Warning: Logits are not supported for METL models. Setting to False." | ||
| ) | ||
| self.return_logits = False | ||
| if "logits" in self.output_types: | ||
| self.output_types.remove("logits") |
Comment on lines
+47
to
+54
| if self.return_contacts: | ||
| logger.warning( | ||
| "Warning: Attention matrices are not supported for METL models. Setting to False." | ||
| ) | ||
| self.return_contacts = False | ||
| for output_type in ("attention_head", "attention_layer", "attention_model"): | ||
| if output_type in self.output_types: | ||
| self.output_types.remove(output_type) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release 1.5.0 — promotes
testtomain, which triggers the PyPI, Conda, and GHCR Docker publish workflows.Highlights
metl-pretrainedbackend (metl-*-1dmodel ids), withMETLEmbedder/METLDataset, typed errors, and CI-gated tests. 1D embeddings only (per_token,mean_pooled,substring_pooled); logits/attention and 3D models are rejected with clear errors.ghcr.io/csi-greifflab/pepe-cli) ontestandmainreleases. Main releases push:latestand the version tag; release notes includedocker pull/docker runexamples.requires-python >=3.10).[esm]optional dependency extra.Release mechanics
__version__bumped1.4.0 → 1.5.0insrc/pepe/__init__.py(single source of truth).Unreleasedcut into[1.5.0]; restored the Docker/GHCR entry that was dropped in an earlier merge intotest.-dev/-testsuffix, so the main publish workflow will release.🤖 Generated with Claude Code